TensorFlow, KerasとPython3を使って、自然言語処理や時系列データ処理を学びましょう。日本語＋動画で学べる唯一の講座（2017年8月現在）です。RNN/LSTMは、機械翻訳、自動字幕表示、株価予測などに使用されています。

We fell for Recurrent neural networks (RNN), Long-short term memory (LSTM), and all their variants. Now it is time to drop them! It is the year 2014 and LSTM and RNN make a great come-back from the dead. But we were all young and unexperienced. For a few years this was the way to solve sequence learning, sequence translation (seq2seq), which also resulted in amazing results in speech to text comprehension and the raise of Siri, Cortana, Google voice assistant, Alexa.

Summary: There are some interesting use cases where combining CNNs and RNN/LSTMs seems to make sense and a number of researchers pursuing this. However, the latest trends in CNNs may make this obsolete. There are things that just don't seem to go together. Take oil and water for instance. Both valuable, but try putting them together?

Want to watch this again later? Report Need to report the video? Report Need to report the video? Need to report the video? This feature is not available right now.

This is a recursive neural network (LSTM type) with 3 layers of 680 neurons each, trying to find patterns in audio and reproduce them as well as it can. It's not a particularly big network considering the complexity and size of the data, mostly due to computing constraints, which makes me even more impressed with what it managed to do. The audio that the network was learning from is voice actress Kanematsu Yuka voicing Hinata from Pure Pure. I used 11025 Hz, 8-bit audio because sound files get big quickly, at least compared to text files - 10 minutes already runs to 6.29MB, while that much plain text would take weeks or months for a human to read. I wrote a program that converts any data into UTF-8 text and vice-versa, and to my excitement, torch-rnn happily processed that text as if there was nothing unusual.

This is a recursive neural network (LSTM type) with 3 layers of 680 neurons each, trying to find patterns in audio and reproduce them as well as it can. It's not a particularly big network considering the complexity and size of the data, mostly due to computing constraints, which makes me even more impressed with what it managed to do. The audio that the network was learning from is voice actress Kanematsu Yuka voicing Hinata from Pure Pure. I used 11025 Hz, 8-bit audio because sound files get big quickly, at least compared to text files - 10 minutes already runs to 6.29MB, while that much plain text would take weeks or months for a human to read. I wrote a program that converts any data into UTF-8 text and vice-versa, and to my excitement, torch-rnn happily processed that text as if there was nothing unusual.

I'm getting some practice with RNN and LSTMs right now, and honestly I'm a little afraid of overfitting; It just feels a lot harder to avoid with these models over CNNs or vanilla NNs. For now I'm working with one hidden layer since I don't expect my data to be bigger than perhaps 1.5 MBs of characters. I'm just looking for some opinions on where dropout should be implemented Please correct me if I'm approaching dropout improperly with RNN/LSTMs. My preliminary basic idea was to add dropout on the hidden states (maybe @ 0.5 to safely begin with? I can cross validate the HP likely), leaving the LSTM cell states as is.